Data Partitioning for Networked Parallel Processing

The workstation model of parallel processing presents speciic challenges caused by the latency of the communications network and the workload imbalance that arises from the heterogeneity of the nodes. Data partitioning is critically important for parallel processing in this environment. We mathematically characterize the communication costs for four data decomposition schemes: scatter, contiguous point, contiguous row, and block. These methods are analyzed in terms of problem size, number of processors, network speed, and communication pattern. Bounds are established for the performance of these decomposition schemes that can be used to make better-informed data partitioning decisions.